System and method for fitness evaluation for optimization in document assembly

ABSTRACT

What is disclosed is a system and method for method for fitness evaluation to be used with a directly calculated or iterative optimization method for automatic document assembly. The method for fitness evaluation includes the steps of first capturing the creator&#39;s desire as a set of relative weights to be applied to an intent vector. Then, calculating for each candidate document assembly a set of value-property functions that evaluate properties considered to be factors in a good design (e.g., the balance of the document component on the page). Next, an inferred intent vector needs to be determined for each candidate document assembly as a function of the calculated value property function. Note that if the inferred intent vector is determined by a matrix multiplication applied to the vector of value property function results then the relative weights and intent definition matrix can be multiplied together to provide a weight vector that can be applied directly to the value properties. The desired weights are then applied to the inferred intents to derive a fitness measure by multiplying the intent vector components by weights and summing.

RELATED APPLICATIONS

Attention is directed to commonly and assigned application Nos.

U.S. Ser. No. 10/202,046, filed Jul. 23, 2002 entiled“CONSTRAINT-OPTIMIZATION SYSTEM AND METHOD FOR DOCUMENT COMPONENT LAYOUTGENERATION”.

U.S. Ser. No. 10/202,188, filed Jul. 23, 2002 entiled“CONSTRAINT-OPTIMIZATION SYSTEM AND METHOD FOR DOCUMENT COMPONENT LAYOUTGENERATION”.

U.S. Ser. No. 10/202,183, filed Jul. 23, 2002 entiled “SYSTEM AND METHODFOR CONSTRAINT-BASED DOCUMENT GENERATION”.

U.S. Ser. No. 10/202,275, filed Jul. 23, 2002 entiled “SYSTEM AND METHODFOR CONSTRAINT-BASED DOCUMENT GENERATION”.

U.S. Ser. No. 10/202,207, filed Jul. 23, 2002 entiled “SYSTEM AND METHODFOR DYNAMICALLY GENERATING A STYLE SHEET”.

U.S. Ser. No. 10/202,247, filed Jul. 23, 2002 entiled “SYSTEM AND METHODFOR DYNAMICALLY GENERATING A STYLE SHEET”.

U.S. Ser. No. 10/202,227, filed Jul. 23, 2002 entiled “CASE-BASED SYSTEMAND METHOD FOR GENERATING A CUSTOM DOCUMENT”.

U.S. Ser. No. 10/202,047, filed Jul. 23, 2002 entiled “CASE-BASED SYSTEMAND METHOD FOR GENERATING A CUSTOM DOCUMENT”.

FIELD OF THE INVENTION

The present invention is directed to systems and methods to finddocument components and assemble them into a custom document such as avariable data document and, in particular, those systems and methodswhich use constraint-optimization approaches wherein the document, itscontent, components, and its requirements are expressed as a constraintoptimization problem.

BACKGROUND OF THE INVENTION

Custom documents are documents that are personalized or tailored in someway to the particular user of the document. Two growing applications ofcustom documents are in the domain of variable data printing, as well asin web personalization.

Traditional approaches to custom document creation are non-automated andtherefore user-intensive, and result in documents that are typicallyquite similar: the layout is the same for all instances, regardless ofthe available content pieces. Furthermore, the document creator isresponsible for ensuring that the final document adheres to good designprinciples, and is therefore aesthetically pleasing. Thus the documentcreator himself typically creates the document template according to hispreferred design criteria, which requires knowledge about documentdesign and how to best achieve the desired qualities in a particularinstance of the document.

Traditional creation of custom documents such as variable data documentsrequires expertise in many areas such as graphic arts and databases andis a time consuming process. With the ever-increasing amount ofinformation in the digital world and the amount of untrained usersproducing documents, old publishing tools often prove cumbersome anddemanding whereas present dynamic digital environments demand tools thatcan reproduce both the contents and the layout automatically tailored topersonal needs and which can enable novices to easily create suchdocuments.

Known methods for automated creation of documents have focused more onparticular types of documents, and not on modeling the problem in ageneral way in order to address all types of documents. Existing workprovides methods for creating diagrams ( see Dengler, E. Friedell, M.,Marks, J., Constraint-Driven Diagram Layout, Proceedings of the 1993IEEE Symposium on Visual Languages, pages 330–335, Bergen, Norway,1993), or multimedia presentations (see Rousseau, F., Garcia-Macias, A.,Valdeni de Lima, J., and Duda, A., User Adaptable MultimediaPresentations for the WWW, Electronic Proceedings from the 8^(th)International World Wide Web Conference, 1999), or flowcharts and yellowpages (see Graf, W. H., The Constraint-Based Layout Framework LayLab andApplications, Electronic Proceedings of the ACM Workshop on EffectiveAbstractions in Multimedia, 1995). Others have explored automating theprocess of web document layout (see Kroener, A., The Design Composer:Context-Based Automated Layout for the Internet, Proceedings of the AAAIFall Symposium Series: Using Layout for the Generation, Understanding,or Retrieval of Documents, 1999).

Known methods for a constraint-optimization approaches to documentlayout use a single optimization criterion: cost, and model their layouttask as finding an ordering of stories and advertisements that canminimize the production cost as described in U.S. Pat. No. 6,173,286.The present invention differs in that it offers a more general model forrepresenting a layout problem as a constraint optimization problem,enables the specification of multiple optimization criteria, andprovides a process by which to combine required and optimizationconstraints in order to achieve a well-designed document.

What is needed in the arts in order to ensure that an automaticallyassembled document also meets desired aesthetic design criteria, is away to model document creation as a multi-criteria optimization problem,allowing the specification of both required layout constraints as wellas desired aesthetic qualities of the output document, and a means toautomatically process this combination of hard and soft constraints toautomatically generate a well-designed document.

SUMMARY OF THE INVENTION

What is disclosed is a system and method for method for fitnessevaluation to be used with a directly calculated or iterativeoptimization method for automatic document assembly. The method forfitness evaluation includes the steps of first capturing the creator'sdesire as a set of relative weights to be applied to an intent vector.Then, calculating for each candidate document assembly a set ofvalue-property functions that evaluate properties considered to befactors in a good design (e.g., the balance of the document component onthe page). Next, an inferred intent vector needs to be determined foreach candidate document assembly as a function of the calculated valueproperty function. Note that if the inferred intent vector is determinedby a matrix multiplication applied to the vector of value propertyfunction results then the relative weights and intent definition matrixcan be multiplied together to provide a weight vector that can beapplied directly to the value properties. The desired weights are thenapplied to the inferred intents to derive a fitness measure bymultiplying the intent vector components by weights and summing. Agenetic algorithm can be used as the iterative optimization methodwherein each candidate document assembly is described as a genome andthe fitness measure is then calculated for each genome and is used indetermining its survival.

Other objects, advantages, and salient features of the invention willbecome apparent from the detailed description which, taken inconjunction with the drawings, disclose the preferred embodiments of theinvention.

DESCRIPTION OF THE DRAWINGS

The preferred embodiment and other aspects of the invention will becomeapparent from the following detailed description when taken inconjunction with the accompanying drawings which are provided for thepurpose of describing the invention and not for the limitation thereof,in which:

FIG. 1 illustrates a document template which specifies that there aretwo areas that should be filled with content: areaA and areaB, and whichalso specifies that the positions and sizes of areaA and areaB can bechanged; and

FIG. 2 illustrates the resulting genome after following through theexample of FIG. 1.

DETAILED DESCRIPTION OF THE INVENTION

What is disclosed is a system and method for specifying a customdocument as a constraint optimization problem, and a method toautomatically create the specified document using one of a set of manyexisting constraint optimization algorithms. The document is modeled asa constraint optimization problem which combines both requiredconstraints with non-required design constraints that act asoptimization criteria. One of a set of many existing constraintoptimization algorithms is then used to solve the problem, resulting inan automatically generated document that is well designed because it hasoptimized some specified design criteria.

In particular, a document template is represented as a constraintoptimization problem, and therefore contains a set of variables, a valuedomain for each variable, a set of required constraints, and a set ofdesired constraints (i.e. optimization functions).

In this invention, the areas of the document to be filled with contentare modeled as problem variables, as are any parameters of the documentthat can be changed. As an example, consider the document template shownin FIG. 1. The template specifies that there are two areas that shouldbe filled with content: areaA and areaB. The template also specifiesthat the positions and sizes of areaA and areaB can be changed. Thus,the problem variables for this example are: areaA, areaB,areaA-topLeftX, areaA-topLeftY, areaB-topLeftX, areaB-topLeftY,areaA-width, areaA-height, areaB-width, areaB-height.

The constraint optimization formulation further specifies that eachproblem variable has a value domain consisting of the possible values toassign to that variable. This invention teaches that for variables thatare document areas to be filled with content (e.g., areaA and areaB ofFIG. 1), the value domains are the content pieces that are applicable toeach area. For variables that are document parameters, the value domainsare discretized ranges for those parameters, so that each potentialvalue for the parameter appears in the value domain e.g., 1..MAXINT. Forvariables whose value domains are content pieces, the default domain isset up to be all possible content pieces in the associated contentdatabase, which is specified in the document template.

The required constraints specify relationships between variables and/orvalues that must hold in order for the resulting document to be valid.The desired constraints specify relationships between variables and/orvalues that we would like to satisfy, but aren't required in order forthe resulting document to be valid. Constraints may be unary (apply toone value/variable), binary (apply to two values/variables), or n-ary(apply to n values/variables), and in our invention are entered by theuser as part of the document template. An example of a required unaryconstraint in the document domain is: areaA must contain an image of acastle. An example of a required binary constraint is:areaA-topLeftY+areaA-height<areaB-topLeftY. If we had another variable(areaC), an example of a required 3-ary constraint is:areaA-width+areaB-width>areaC-width. In a variable data application ofthis invention (one of many possible applications), the constraintswould also refer to customer attributes (e.g., areaA must contain animage that is appropriate for customer1.age).

Desired constraints are represented as objective functions to maximizeor minimize. For example, a desired binary constraint might be theobjective function: f=areaA-width*areaA-height, to be maximized. If morethan one objective function is defined for the problem, the problembecomes a multi-criteria optimization problem. If it is a multi-criteriaoptimization problem, we sum the individual objective function scores toproduce the overall optimization score for a particular solution. We canfurthermore weight each of the desired constraints with a priority, sothat the overall optimization score then becomes a weighted sum of theindividual objective function scores.

Any one of the known existing constraint optimization algorithms is thenapplied to create the final output document. This invention furtherdescribes a means to use a genetic algorithm (one of the many possibleconstraint optimization algorithms) for doing the constraintoptimization and thereby automatically creating a final output documentthat adheres not only to the required constraints, but also to a set ofdesired constraints.

In our genetic algorithm formulation of constraint optimization fordocument creation, the genome is built such that each gene in the genomeis a variable of the constraint problem. Following through our examplefrom FIG. 1, the resulting genome is shown in FIG. 2. The unaryconstraints are used to set up the allowable value domains for eachgene. These can be some default range, or input by the user.

In this invention, the fitness function is defined such that it returnsa fitness of 0 for any population members that do not meet the requiredconstraints, and for the members that do meet the required constraints,it returns a fitness score that is a sum of the scores of the individualdesired constraints. For instance, if we have the required constraints:

C1: areaA-width<300

C2: areaB-width<300

And the desired constraints:

C3: areaA-width=areaB-width, to be maximized (ranges from 0 to 1)

C4: areaA-height=areaB-height, to be maximized (ranges from 0 to 1)

Examples of fitness function for these desired constraints aref3=1-|areaA-width−areaB-width|/(areaA-width+areaB-width)f4=1-|areaA-height−areaB-height|/(areaA-width+areaB-height)

If we have a population member with areaA-width=350, areaA-height=350,areaB-width=400, areaB-height=200, the fitness function returns a scoreof 0. If, however, we have a population member with areaA-width=300,areaA-height=200, areaB-width=300, areaB-height=200, the fitnessfunction returns a score of 2. If we have a population member withareaA-width=225, areaA-height=200, areaB-width=300, areaB-height=200,the fitness function returns a score of 1.875.

Our formulation also extends to allow weighting of the various desiredconstraints. Thus, the document creator can specify that certain desiredconstraints are more important than others. For instance, we could haveconstraint C3 weighted with an importance of 1.5, and C4 weighted withan importance of 0.5, meaning that the two objects having the same widthis more important than the two objects having the same height. Thefitness function's overall score is then computed as a weighted sum ofthe individual desired constraints.

For instance, if we have a population member with areaA-width=225,areaA-height=200, areaB-width=300, areaB-height=200, desired constraintC3 returns 0.875, which is multiplied by C3's weight of 1.5, to get1.286. Desired constraint C4 returns 1, which is multiplied by C4'sweight of 0.5, to get 0.5. The overall fitness score is then1.125+0.5=1.786.

If, on the other hand, we have a population member with areaA-width=300,areaA-height=200, areaB-width=300, areaB-height=150, desired constraintC3 returns 1, which is multiplied by C3's weight of 1.5 to get 1.5 .Desired constraint C4 returns 0.875, which is multiplied by C4's weightof 0.5, to get 0.438. The overall fitness score is then 1.5+0.438=1.938,thereby preferring the solution that violates C3 the least.

In the genetic algorithm implementation of this invention, we create aninitial population of chromosomes by selecting values for each gene, anddoing this for the desired number of population members. We evaluateeach member of this population according to the fitness function,resulting in a score for each population member. We then select the mostfit individuals (i.e., best fitness score) as parents for the newpopulation, and create a new population from the parents usingcrossover/mutation operations. We iterate through populations until wereach a specified stopping condition (e.g., a certain number ofiterations are complete, or until we have crossed a minimum thresholdfor the fitness function).

Thus, each genome is evaluated according to how well it satisfies orachieves the design qualities along with the other required constraints.This results in a generated document that not only satisfies therequired constraints, but that is also optimized for the specifieddesign qualities.

Further regarding fitness evaluation, during the creation of a documentthe document creator makes many decisions (e.g., what size font to use,what type of font, how long the lines should be, etc.) where presumablythese decisions are made in order to achieve certain value properties inthe document (e.g., low cost, balanced, readable, etc.). There are manysuch properties to consider when creating a document, and thus the needfor many decisions by the author. The intent lies in the relativeimportance of the various value properties. Depending on intent, certainproperties will be strengthened while others will be sacrificed. Intentinformation to optimize document creation and formatting was explored.The idea that intents could be used to define a linear combination ofmeasured value properties. This can be expressed as: I=A V where I is avector of intent coordinates inferred from the properties of thedocument, V is a vector of value properties and A is a matrix relatingthe value properties to intents. This suggests that document propertiescould be adjusted to produce value properties that in turn generate adesired intent. Thus, f₁=|I−I_(d)| is a fitness function that tells howwell the document matches the desired intent I_(d). Minimizing thisfunction can be used to generate the document or its presentation.However, this does not form a document solution with value propertiesleading to intent values greater than the original desired intent I_(d).In some cases, it may be the relative strengths of the intents that isimportant rather than their absolute measure. Thus an alternativefitness function utilizing the weighted sum of the inferred intents ispreferred and can be defined as: f₂=w I=w A V where w is the desiredproportion or weighting of the intents. Maximizing this function allowsthe various intents to be maximized but when increasing one intentresults in a decrease of another intent then the weighting factorscontrol the final proportions chosen. This fitness function willoptimize value properties and use the intent weights to select whichproperty to optimize when increasing one decreases another. In addition,a genetic algorithm can be used where the fitness function is a weightedsum of the desired properties of the document. Further, since genomesdefine document layouts and the genetic algorithm determines whichgenomes survive by evaluating their fitness then, by using the fitnessfunction of the genetic algorithm, the generated solutions become closerand closer to the creator's intent.

The method for fitness evaluation includes the steps of first capturingthe creator's desire as a set of relative weights to be applied to anintent vector. Then, calculating for each candidate document assembly aset of value-property functions that evaluate properties considered tobe factors in a good design (e.g., the balance of the document componenton the page). Next, an inferred intent vector needs to be determined foreach candidate document assembly as a function of the calculated valueproperty function. Note that if the inferred intent vector is determinedby a matrix multiplication applied to the vector of value propertyfunction results then the relative weights and intent definition matrixcan be multiplied together to provide a weight vector that can beapplied directly to the value properties. The desired weights are thenapplied to the inferred intents to derive a fitness measure bymultiplying the intent vector components by weights and summing. Agenetic algorithm can be used as an iterative optimization methodwherein each candidate document assembly is described as a genome andthe fitness measure is then calculated for each genome and is used indetermining its survival.

The system and method of the present invention has many advantages overthe prior art. Whereas the current constraint satisfaction approachesoften require many low-level layout constraints to be specified in orderto achieve a reasonable result, the genetic algorithm approach disclosedherein allows a specification of a few high-level desired constraintsand qualities—a much more intuitive and less user-demanding process.Another advantage of the constraint optimization approach describedherein is that it can find pleasing solutions for any combination ofcontent thereby enabling more dynamic custom document instances. Inaddition, selection of content can be influenced by the design criteriathat is included in the solving process by creating genes that specifythe number of items to include for each content area and, as the genevalue varies, the content items included vary as well. Another advantageof the present constraint-optimization system and method is that thevarious aesthetic criteria can be weighted and result in a differentoutput document based on the weightings (e.g., a different outputdocument would be generated if compactness was heavily weighted than ifpage utilization was heavily weighted).

While the invention is described with reference to a particularembodiment, this particular embodiment is intended to be illustrative,not limiting. Various modifications may be made without departing fromthe spirit and scope of the invention as defined in the amended claims.Modifications and alterations will occur to others upon reading andunderstanding this specification; therefore, it is intended that allsuch modifications and alterations are included insofar as they comewithin the scope of the appended claims or equivalents thereof.

1. A method for evaluating, by generating a fitness measure value, a setof variable data documents generated by an automatic document assemblyprocess, a set of variable data documents being a set of documentshaving a portion corresponding to a predetermined content and a portioncorresponding to a variable content, the predetermined content being thesame in each document of the set of variable data documents, comprising:(a) inputting, through an input device, document specifications for aset of variable data documents to be generated, the documentspecifications being represented as a set of relative weights; (b)generating, using a processor, an set of electronic variable datadocuments; (c) executing,for each electronic variable data document, aset of value-property functions to generate a set of value properties,said set of value-property functions evaluating properties representinga good design; (d) determining an inferred intent vector for eachelectronic variable data document as a function of the set of calculatedset of value properties, the inferred intent vector is determined by amatrix multiplication applied to a vector of value properties; and (e)generating a fitness measure value by multiplying components of theinferred intent vector by a corresponding relative weight from the setof relative weights to generate a set of products and summing the set ofproducts.